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1.  BACKGROUND 


The  first  known  regulation  to  control  a  profession  by  specifying  a  professional  license 
was  established  in  1 140  A.D.  when  King  Roger  of  Normandy  decreed  that  all  physicians  present 
proof  of  competency  before  being  allowed  to  practice  medicine.  After  successfully  completing 
all  required  academic  work  and  a  written  comprehensive  examination,  the  physician  would  be 
certified  to  provide  medical  services  to  the  public. 

In  the  early  1760’s  John  Smeaton  identified  himself  as  the  first  “civil  engineer”  to 
distinguish  himself  from  the  “military”  engineers  and  to  clearly  designate  his  target  market.  In 
1852,  the  United  States  began  recognizing  civil  engineering  through  the  establishment  of  the 
American  Society  of  Civil  Engineers  (ASCE).  At  that  time  membership  in  ASCE  was  sufficient 
to  be  recognized  as  a  professional  engineer  by  the  courts.  During  the  1 870’s  and  1880’s,  a 
number  of  railroad  bridge  failures  occurred,  resulting  in  a  tremendous  loss  of  life.  Many  of  these 
failures  were  consistent  with  the  evolving  nature  of  the  engineering  profession  that  included  new 
materials,  innovative  construction  systems  and  more  complex  load  distributions  associated  with 
longer  span  bridges.  The  public  attributed  the  accidents  to  incompetent  and  unqualified 
engineers.  This  led  to  the  heed  for  legal  registration  of  the  profession  in  1897. 

In  1907,  after  much  discussion,  a  panel  of  1 1  ASCE  members  in  Wyoming  established 
the  first  licensing  law  governing  the  practice  of  engineering  and  surveying.  After  the  bill  was 
passed,  written  examinations  began  and  within  2  years,  134  engineers  were  registered  in 
Wyoming.  Other  states  soon  enacted  their  own  legislation  and  by  1947,  every  state  had 
legislation  controlling  the  registration  of  engineers;  however,  licenses  granted  by  one  state  were 
not  recognized  by  another  states.  In  1920,  the  Council  of  State  Boards  for  Engineering 
Examiners  was  established,  and  later  renamed  to  the  National  Council  of  Engineering  Examiners 
(NCEES)  in  1967,  in  an  attempt  to  promote  more  uniform  licensing  standards,  national  licensure, 
and  reciprocity  between  the  states.  Although  each  State  maintains  specific  individual  licensing 
requirements,  all  states  use  the  exam  prepared  by  the  NCEES  to  ensure  each  examinee  is 
provided  with  the  same  opportunity  to  become  licensed. 
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The  licensing  requirements  for  Civil  Engineering  include  documentation  of  relevant  work 
experience,  personal  references,  and  a  written  exam.  The  original  licensure  exam  involved  a  4- 
hour  written  problem  solving  section,  which  allowed  the  examinee  to  receive  partial  credit.  Due 
to  the  non-uniform  and  subjective  in-state  grading  practices,  the  NCEES  changed  the  exam 
format.  The  current  NCEES  exam  format,  which  began  in  October  2001,  for  a  general  Civil 
Engineering  exam  consists  of  an  4-hour  multiple  choice  “depth”  exam  concentrating  on  one  of 
the  5  major  topic  areas:  geotechnical,  environmental,  structural,  transportation,  water  systems; 
following  a  4-hour  multiple  choice  “breadth”  exam  that  covers  all  areas  of  Civil  Engineering 
(20%  of  the  questions  from  each  area). 

According  to  data  gathered  from  the  NCEES  an  estimated  23,500  Civil  Engineers  register 
to  take  the  Professional  Engineer  (P.E.)  licensure  exam  each  year.  Approximately  400  of  these 
examinees  apply  to  take  the  exam  in  North  Carolina.  Study  and  preparation  for  the  P .E.  exam  is 
a  major  concern  for  engineers  seeking  their  license.  Most  P.E.  candidates  are  practicing 
engineers  that  depend  on  a  variety  of  study  resources  and  refresher  classes  due  to  the  “breadth” 
of  the  examination 

The  Industrial  Extension  Service  (IES)  of  North  Carolina  State  University  is  a  provider  of 
a  high  quality  P.E.  refresher  course  which  is  taught  in  a  traditional  live  classroom  format.  The 
IES  is  currently  considering  an  expansion  of  the  P.E.  refresher  course  by  means  of  utilizing  a 
suitable  distance  learning  method  to  reach  a  larger  audience  while  maintaining  a  high  quality  of 
P.E.  training  offerings. 

In  a  survey  done  by  the  IES  in  early  2001,  31  of  32  engineers  preparing  for  the  exam 
purchased  some  sort  of  study  material.  Of  these  32,  17  attended  a  live  training  course,  but  28 
would  consider  a  distance  learning  course.  Additionally,  23  of  these  28  would  purchase  a  CD- 
ROM  and  textbook  distance  learning  course  if  the  price  and  quality  was  competitive  to  a  live 


course. 


2.  PROBLEM  STATEMENT 


Engineers  seeking  to  obtain  Professional  Engineer  licensure  typically  attend  a  formal 
classroom  refresher  course  to  help  prepare  for  the  written,  qualification  exam,  especially  those 
practicing  abroad.  Due  to  the  flexibility  to  meet  the  constraints  of  practicing  engineers,  distance 
learning  methods  have  become  useful  tools  to  assist  in  preparation  for  the  Professional  Engineer 
exam.  Distance  learning  formats  include  video  lectures,  live  online  classes,  and  self-paced  CD- 
ROMs.  As  a  result  of  continually  improving  technology,  distance  learning  methods  have 
become  more  complete  and  affordable.  The  survey  by  IES  also  indicated  that  engineers  would 
strongly  consider  distance  learning  methods  to  prepare  for  the  P.E.  exam. 

As  a  provider  of  a  “traditional”  P.E.  review  course,  the  IES  requires  a  quick  survey  of 
different  existing  distance  learning  delivery  methods  for  the  P.E.  review.  Several  currently 
available  Civil  Engineering  P.E.  refresher  distance  learning  courses  were  examined  and  the  one 
“best”  alternative  was  selected  for  an  in-depth  analysis  to  enable  IES  to  offer  a  P.E.  review 
course  via  distance  learning.  The  in-depth  subject  evaluation  of  the  selected  course  was 
completed  from  the  point  of  view  of  a  “typical”  engineer  preparing  for  the  exam  to  identify  the 
advantages  and  disadvantages  of  the  course  and  to  assist  IES  in  the  development  of  a  separate, 
independent  distance-learning  course. 

This  report  summarizes  the  various  P.E.  distance  learning  courses  currently  available, 
describes  the  selection  criteria  for  evaluating  one  specific  course,  and  discusses  the  advantages 
and  disadvantages  of  the  course.  Recommendations  are  provided  for  the  IES  for  possible 
implementation  and  further  study. 
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3.  OBJECTIVES 


3.1  General 

Obtaining  P.E.  Licensure  is  a  defining  milestone  in  an  engineer’s  career.  The  traditional 
method  of  preparing  for  the  written  exam  involves  a  self-directed  or  classroom  refresher  course. 
Self-directed  studies  lack  structure  and  interaction.  Traditional  classroom  courses  lack  flexibility 
to  meet  the  working  schedule  of  a  typical  engineer  and  have  limited  attendance  due  to 
constrained  geographic  location.  With  rapidly  developing  technology,  several  different  distance 
learning  courses  become  available  which  offers  and  attractive  alternative  to  the  “traditional” 
means  of  examination  review. 


3.2  Project  Objectives 

The  objectives  of  this  project  are  as  follows: 

•  Identify  critical  factors  that  would  make  distance  learning  courses  a  desirable 
alternative 

•  Identify  various  representative  distance  learning  P.E.  review  courses  currently 
available 

•  Select  a  “best”  course  alternative 

•  Evaluate  the  selected  course  and  identify  advantages  and  disadvantages 

•  Develop  break  even  cost  evaluation  for  product  development 

•  Provide  feedback  to  the  IES  for  future  development  of  a  distance  learning 
course 

3.3  Limitations 

This  report  was  completed  with  the  following  limitations: 


•  Completeness  /  Exhaustiveness:  Not  all  available  P.E.  refresher  courses  were 
investigated.  Only  representative  courses  that  were  found  via  an  internet  search  were 
considered  for  the  initial  sample  of  distance  learning  courses. 

•  Obiectiveness:  The  analysis  of  the  evaluated  course  is  subjective  and  represents  the 
view  of  a  practicing  engineer  preparing  for  the  P.E.  exam.  This  objectiveness  also 
applies  to  the  identification  of  relative  weight  factors  and  other  non-scientific  criteria 
used  for  evaluation  purposes. 

•  Subjective  Ranking:  All  rankings  were  based  on  the  perception  and  subjectivity  of 
the  evaluating  engineer  and  were  not  a  measure  of  the  course  content  quality. 
Rankings  were  used  to  produce  a  relative  measure  of  completeness,  through  a 
desirable  product  matrix,  for  future  product  development  purposes. 
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4.  METHODOLOGY 


4.1  General 

This  study  was  conducted  in  three  phases.  Phase  I  consisted  of  identifying  a  sample  of 
available  distance  learning  courses  and  developing  a  product  evaluation  matrix  that  was  based  on 
the  course  description  as  advertised  by  the  seller.  Phase  II  involved  selecting  and  performing  a 
detailed  evaluation  of  one  course.  Phase  HI  consisted  of  writing  a  report  with  specific 
recommendations  for  development  of  a  distance  learning  course  by  IES. 

All  phases  of  this  report  were  conducted  by  an  engineer  (“the  evaluator”)  that  meets  the 
profile  of  a  typical  examinee  likely  to  use  a  distance  learning  course  to  prepare  for  the  P.E. 
examination.  The  evaluator  was  a  graduate  student  completing  a  Master’s  Degree  in  Civil 
Engineering.  The  evaluator  had  previously  obtained  a  Bachelor’s  degree  in  Aerospace 
Engineering  from  a  4  year  ABET  accredited  university.  The  student  had  met  all  experience 
requirements  for  the  P.E.  exam  and  had  6  months  to  prepare  for  the  Civil  Exam. 

4.2  Phase  I:  Course  Identification 

Available  distance  learning  refresher  courses  were  identified  through  an  extensive 
Internet  keyword  search.  Additionally,  the  National  Society  of  Professional  Engineers  (NSPE), 
the  American  Society  of  Civil  Engineers  (ASCE),  and  the  National  Council  of  Examiners  for 
Engineering  and  Surveying  (NCEES)  were  contacted  to  identify  additional  courses.  The  NCEES 
administers  exams  throughout  the  country  but  does  not  provide  refresher  courses.  Contacts  with 
NSPE  and  ASCE  did  not  yield  course  offerings  different  from  those  already  identified.  A 
representative  list  of  currently  available  P.E.  Review  courses  is  outlined  in  Appendix  A,  along 
with  corresponding  course  format  and  contact  information. 
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4.3  Phase  II:  Course  Selection 


The  primary  types  of  distance  learning  course  formats  are  classroom,  textbook,  video, 
and  CD-ROM.  Each  of  these  formats  can  be  presented  in  two  ways:  online  and  offline  courses. 
Online  courses  involve  the  use  of  the  internet  to  interact  with  others  using  one  of  the  four 
methods.  The  reference  material  from  an  online  textbook  course  could  be  downloaded  from  the 
internet.  Offline  courses  concentrate  on  self-paced  learning  that  is  not  associated  with  the 
internet.  The  advantages  and  disadvantages  of  each  method  and  format  are  listed  in  Table  1 . 


Table  1:  Distance  Learning  Course  Format  Summary 


f  Format 

Advantage 

fill 

Disadvantage 

Classroom 
(On-line  & 
Offline) 

Face-to-Face  Contact 

Scheduling  Difficulties 

Real  Time  Feedback  /  Assistance 

Extensive  Instructor  Involvement 

Group  Discussion  Easily  Conducted 

Live  Internet  Course  Expensive  to 
Administer 

Lack  of  Flexibility  in  Attendance 

Review  Course  Lengths  typically 
longer  than  other  formats 

Textbook 
(On-line  & 
Offline) 

Good  Reference  to  Take  Into  the  P.E. 
Exam 

No  Student  /  Teacher  Interaction 

Larger  Variety  of  Practice  Problems 

No  Feedback 

All  Material  is  Self-Taught  with  No 
Instruction 

CD-ROM 
(On-line  & 
Offline) 

Ability  to  See  the  Instructor  on  Pre- 
Recorded  Lesson 

Minimal  Feedback 

Minimal  Instructor  Involvement  with 
Course  after  Initial  Development 

Minimal  Student  /  Teacher  Interaction 

Printable  Text  Available  for  Reference 
during  P.E.  Exam 

Limited  Variety  of  Problems 

High  Quality  Visual  Aids 

Lesson  Topic  Instruction  Quick  and 
Easy  to  Repeat 

Video 

(Offline) 

Ability  to  See  Instructor 

No  Feedback 

Minimal  Instructor  Involvement  with 
Course  after  Initial  Development 

No  Student  /  Teacher  Interaction 

No  Written  Text  or  Example  Problems 
Provided 

Review  of  Specific  Topics  are  Slow  to 
Repeat 
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The  online,  live  classroom  environment  is  most  effective  in  allowing  people  to  interact 
and  discuss  problems  and  solutions.  This  format  is  costly  to  produce  over  the  internet  and 
requires  specific  scheduling  by  students.  The  textbook  is  a  very  common  method  of  review  and 
provides  a  useful  reference  that  can  be  brought  into  the  exam.  Use  of  a  textbook  is  designed  for 
self-paced  learning  without  the  benefit  of  personnel  interaction.  CD-ROM  and  video  courses 
combine  the  advantages  and  disadvantages  of  classroom  and  textbook  distance  learning  formats 
into  a  useful  and  effective  course.  The  advantages  of  each  method  and  delivery  mechanism  were 
integrated  into  a  more  formal  evaluation  procedure  to  select  a  “best”  course  to  meet  IES 
requirements.  Full  discussion  of  the  evaluation  procedure  is  provided  in  Chapter  5. 

4.4  Phase  H:  Course  Evaluation 

A  comparison  chart  was  first  developed  to  summarize  the  advantages  and  disadvantages 
of  each  module  in  the  evaluated  course  and  create  a  set  of  “ideal”  course  characteristics.  Several 
references  were  used  to  identify  key  success  factors  that  contribute  to  the  development  of  an 
effective  learning  environment.  A  hypothetical  “ideal”  learning  environment  was  developed  by 
combining  these  key  factors  into  a  single  comparison  chart.  The  effectiveness  of  any  training 
course  is  dependent  on  the  individual  needs  of  the  student,  and  the  proper  implementation  and 
combination  of  these  keys  factors  should  improve  the  learning  capability  of  an  average  student. 

A  value  of  100%  was  assigned  to  each  factor  to  establish  a  baseline  value  for  comparison. 
Each  key  factor  was  assigned  a  weight  factor  of  importance  based  on  the  similarities  between  the 
references  and  the  level  of  importance  as  based  on  the  evaluator’s  judgement.  These  weight 
factors  were  used  to  determine  an  overall  rating  of  the  individual  modules  compared  to  a 
hypothetical  “ideal”  learning  environment.  The  completed  comparison  chart  is  shown  in 
Appendix  B. 

Following  the  creation  of  the  “ideal”  course  characteristics  chart  shown  in  Appendix  B, 
the  evaluator  enrolled  and  completed  individual  sections  of  the  course.  Upon  completion  of  each 
individual  module,  the  positives  and  negatives  of  each  section  were  compared  with  the  ideal 
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course  characteristics  in  Appendix  B,  and  a  rating  was  assigned  to  each  characteristic  as 
compared  to  the  pre-determined  baseline.  The  individual  module  ratings  were  based  on  how 
effectively  the  module  met  or  did  not  meet  the  selection  criteria.  After  completing  the  matrix  for 
each  module,  the  individual  ratings  were  multiplied  by  its  weight  factor  and  combined  to 
determine  and  overall  rating  for  the  course. 

After  evaluating  each  module  and  completing  the  pre  and  post-exams  offered  with  the 
course,  recommendations  were  developed  which  identified  existing  and  additional  topical 
coverage  areas,  time  to  cover  each  of  the  respective  topic  areas,  presentation  format,  and  types  of 
example  problems  to  present.  In  addition,  modifications  appropriate  for  development  of  a  new 
course  offering  by  IES,  including  the  incorporation  of  new  technologies,  were  recommended. 
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5.  COURSE  SELECTION  CRITERIA 


5.1  General 

Course  selection  criteria  were  established  based  on  on-line  marketing  information 
provided  by  the  product  seller.  A  total  of  7  common  product  features  were  identified  and  are 
summarized  in  Appendix  C.  Each  feature  was  assigned  a  individual  weight  factor  based  on 
perceived  student  needs  and  the  requirements  set  forth  by  the  IES.  The  combination  of  all  seven 
factors  gave  a  total  possible  100  points.  A  full  discussion  of  each  contributing  factor  is  provided 
in  sections  5.2  through  5.8. 

5.2  Course  Format 

The  Industrial  Extension  Service  desires  to  develop  a  new  course  that  captures  the  quality 
of  their  live  course  and  instructors,  yet  is  self-sustaining  in  the  sense  that  it  requires  minimal 
post-development  instructor  involvement.  Presentation  format  is  a  critical  factor  in  determining 
which  course  to  select.  A  live  Internet  course  most  closely  represents  a  classroom  environment 
and  allows  student  interaction;  however,  it  would  require  continuous  instructor  involvement  and 
is  more  costly  than  other  methods.  There  are  numerous  stand-alone  textbooks  available  which 
have  saturated  the  marketplace  and  made  it  difficult  for  a  new  book  to  have  a  competitive 
advantage  over  others. 

Both  the  on-line  and  offline  CD-ROM  and  video  based  formats  were  the  most  applicable 
to  this  project.  Since  the  Industrial  Extension  Service  currently  produces  a  video  based  refresher 
course  which  covers  some  of  the  major  topics  on  the  P.E.,  the  on-line  CD-ROM  format  was 
selected  as  the  most  appropriate  due  to  its  lower  cost  and  flexible  scheduling.  Based  on  the 
importance  of  selecting  the  correct  format,  this  factor  was  assigned  a  maximum  weight  factor  of 
20  out  of  100. 
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5.3  Availability  of  Instructor 


Inevitably  questions  arise  from  difficult  and  unclear  concepts,  or  details  that  were  not 
addressed  in  the  lecture.  The  type  of  assistance  and  the  availability  of  the  instructor  became  the 
second  factor  of  importance.  Only  classroom  or  video-based  courses  that  provide  a  means  of 
interaction  between  the  students  and  instructor,  through  either  email,  telephone,  or  face-to-face 
contact,  were  considered  acceptable  courses.  This  factor  was  considered  critical  in  the 
effectiveness  of  a  distance  learning  course  and  was  also  assigned  a  maximum  weight  factor  of  20 
out  of  100. 

5.4  Class  Duration 

Typically,  individuals  preparing  for  the  P.E.  exam  through  the  use  of  a  classroom  based 
refresher  course  begin  reviewing  material  a  few  months  prior  to  the  exam  date.  Self  directed 
refresher  courses  are  often  completed  in  less  time  than  a  structured  class.  Designing  a  new 
structured  course,  but  on  a  self-paced  level,  caused  the  course  completion  time  to  be  the  third 
factor  of  importance.  The  time  required  to  complete  any  course  is  partially  dependent  on  the 
individual  and  on  the  designed  course  length.  Courses  evaluated  in  this  study  ranged  from  50 
hours  of  self-paced  instruction  to  6  months  of  interactive  class  work.  Since  the  typical  individual 
who  will  be  completing  the  review  course  will  be  maintaining  a  full  time  job,  ideally,  the  shorter 
amount  of  time  required  for  review,  the  more  likely  the  individual  is  to  choose  a  specific  course. 
With  a  wide  range  of  available  course  durations,  this  factor  was  assigned  a  maximum  weight 
factor  of  15  out  of  100. 

5.5  Instructor’s  Knowledge  of  Specific  Exam  Content 

Before  selecting  any  course  of  study,  it  is  important  to  identify  the  instructor’s 
qualifications.  Some  instructor’s  are  unfamiliar  with  the  exact  contents  of  the  P.E.  exam  or  offer 
only  selected  areas  of  instruction.  All  programs  and  vendors  that  produced  a  comprehensive, 
self-sustaining  P.E.  refresher  course  were  consider  for  the  evaluation.  Additionally,  a  program 
that  was  sponsored  by  a  key  engineering  organization  or  a  recognized  educational  institute  were 
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considered  more  favorable  than  non-sponsored  programs  due  to  the  likelihood  of  the 
organization  being  more  familiar  with  the  current  P.E.  exam  content  and  format.  This  factor  was 
assigned  a  maximum  weight  factor  of  15  out  of  100. 

5.6  Cost 

Of  the  courses  identified,  the  costs  ranged  from  $200  for  a  single  instructional  area  to 
$2500  for  an  on-line,  interactive,  comprehensive  refresher  course.  The  cost  was  a  factor  in 
selection  based  on  the  products  marketing  potential.  The  content  of  the  evaluated  product  could 
be  compared  to  the  overall  cost  to  determine  proper  pricing  for  a  similar  course.  The  cost  factor 
was  assigned  a  maximum  weight  factor  of  10  out  of  100. 

5.7  Miscellaneous  Factors 

Appendix  C  summarizes  the  critical  factors  and  weight  factors  used  for  selecting  the 
evaluated  course.  Aside  from  the  previously  discussed  factors,  several  other  non-critical 
selection  factors  were  identified  for  this  study.  The  number  of  lessons  provided  was  considered 
to  be  less  important  than  the  total  time  required  to  complete  the  course;  however  a  sufficient 
number  of  lessons  were  required  to  fully  cover  the  major  topic  areas.  This  caused  the  course 
duration  to  be  weighted  10  out  of  100  points.  Finally,  the  retake  guarantee  would  typically  be  a 
critical  selection  factor  if  cost  was  more  highly  rated.  The  ability  to  retake  the  course  was  not 
weighted  heavily  (10  out  of  100). 

5.8  Selected  Course 

The  course  offered  by  Keepsmart  Engineering,  in  coordination  with  ASCE,  was  selected 
to  be  the  most  applicable  to  this  study  and  received  a  combined  score  of  86  out  of  100.  The 
selected  course  combines  both  a  CD-ROM  and  Internet  format  and  is  designed  to  be  a  7  module, 
50-hour  self-paced  program  covering  each  major  Civil  Engineering  topic. 
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After  purchasing  the  course,  the  evaluator  was  provided  with  immediate  access  to  an 
online  version  of  the  course.  The  course  software  was  then  shipped  overnight  to  the  evaluator. 
The  CD-ROM  combines  both  an  audio  and  video  instructional  presentation,  whereas  the  online 
version  contains  only  audio  clips  due  to  the  large  bandwidth  required  for  video.  In  both 
instances,  help  is  available  for  each  module  through  a  central  point  of  contact  at  Keepsmart 
Engineering  via  email.  A  database  of  questions  is  maintained  at  Keepsmart  Engineering,  and 
any  new  questions  are  forwarded  to  the  instructor  for  evaluation.  Questions  are  then  answered 
via  email  back  to  the  individual.  Feedback  time  for  a  common  question  is  between  30  minutes 
and  24  hours.  In  the  event  the  student  requests  to  discuss  the  topic  or  question  with  the 
instructor,  Keepsmart  Engineering  contacts  the  instructor  to  obtain  approval  for  one-on-one 
contact.  However,  since  the  instructors  are  full  time  engineers,  availability  to  talk  with  them 
personally  is  dependent  on  their  schedule. 
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6.  COURSE  EXAMINATION 

6.1  General  Course  Overview 

The  selected  course  was  divided  into  7  different  modules  that  cover  the  major  topic  areas 
in  Civil  Engineering.  Table  2  lists  all  the  major  topic  areas  covered  by  the  course.  Each  module 
consisted  of  four  sections:  a  pre-exam,  a  demonstration  problem,  a  practice  problem,  and  a  post¬ 
exam.  The  course  moves  through  each  section  of  the  module,  although  the  student  has  the  option 
to  go  directly  to  any  specific  module. 

Table  2:  Course  Content  Summary 


Hydraulics  and 

Hydrology 


SSanitaryiancl,/ 

’Fiji 

Environmental 


Content 


.  ,  _  A  Internal  Rate  of 

Annual  Cost  „ 

_ Return _ 

Depreciation  and  Ranking 

Taxes  Alternative  Projects 


Effective 
Interest  Rate 


Soil 

Classification 


Strength  & 

B earing  Capacity 


Effective  Stress  & 
Stress  Distribution 


Earth  Retaining 
Structures 


Consolidation 


Seepage 


Hydrostatics 


Rainfall  Runoff 


Closed  Circuits  Friction  Losses 


Momentum 


Specific  Energy 


Minor  Form 
Losses 
Manning 
Equation 


Steel  Beams  & 
Trusses 


Beams/Columns 


Concrete  Arches  & 
Columns 


Water  Demand  Water  Softening 


Coagulation  & 
Flocculation 


Concrete 

beams 


Ground  Water 
Problems 


Trickling  Filters 


Stream 

Purification 


Sludge  Digestion  Land  Filling 
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Prior  to  beginning  each  module,  the  evaluator  completed  an  on-line  pre-exam  with  the 
use  of  only  reference  8.  The  pre-exam  was  only  accessible  through  the  online  webpage  and  was 
printed  through  the  use  of  the  “print  screen”  option.  The  evaluator’s  pre-exam  answers  were 
entered  into  the  Keepsmart  Engineering  database  and  used  to  establish  a  subject  matter  baseline 
for  the  evaluator  and  ultimately  measure  the  course  effectiveness  for  that  student. 

After  completing  the  pre-exam,  each  module  presented  a  demonstration  problem 
focusing  on  the  specific  area.  Typically  these  specific  topic  areas  were  similar  to  the  questions 
on  the  pre-exam.  Each  demonstration  problems  was  viewed  and  printed  through  Adobe  Acrobat 
Reader.  The  solutions  for  each  module  were  printed  as  a  whole  with  the  “printable  solutions” 
option,  or  could  be  printed  individually  from  the  specific  subject  demo  problem  screen.  Each 
problem  consisted  of  several  questions,  each  of  which  were  multiple  choice,  and  contained 
embedded  hypertext,  for  example  “manometer”,  “atmospheric  pressure”  and  “head”  as  shown  in 
Figure  1,  that  provided  the  user  with  definitions  and  equations  for  key  words  within  the  problem. 


SITUATION:  1.  Hydrostatics  Demo  Problem 

A  rectangular  panel,  4  feet  wide  and  8  feet  long  Is  held  In  a  tank  at  a  depth  of  9  feet,  parallel  to  the  surface, 
air  U:shaped  manometer  is  tapped!  at  the  bottom  of  the  tank. 

An  open- J|  j 

REQUIREMENT 

[r£in  terms  of  "head"  of  mercury?  (Answer  in  inches: 

(H  5 

' 

C ! 

Is  • 

1  Do  mo  Re  qujrement  d 

1  ii>,4G8 

(11)26  3  W>48 

Mama  1 

1 

Figure  1 :  Example  Demonstration  Problem  Screen 
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During  the  demonstration  problem,  the  evaluator  had  the  option  to  activate  an  instructor 
recording  to  have  the  “instructor”  explain  the  basic  concept  behind  the  problem,  the  problem 
itself,  or  a  specific  problem  requirement  /  question.  In  the  CD-ROM  course,  a  small  screen 
appeared  with  a  video  clip  of  the  instructor  explaining  the  requested  information.  The  online 
course  only  provided  audio  of  the  instructor’s  voice. 

The  third  part  of  the  module  was  the  practice  problem.  Each  module  contained  one 
practice  problem  for  each  major  topic  area.  Each  problem  contained  multiple  questions  with 
each  successive  questions  depending  on  the  previous  one.  The  practice  problem  was  basically  a 
duplication  of  the  demonstration  problem,  but  with  the  numerical  values  changed  as  shown  by 
comparing  Figures  1  &  2.  There  was  no  instruction  provided  to  complete  the  practice  problem. 
The  instructor  non-quantitatively  indicated,  via  audio,  whether  the  evaluator  had  selected  the 
correct  or  incorrect  answer.  The  evaluator  was  continued  selecting  answers  until  the  correct 
answer  was  chosen. 


SITUATION:  1.  Hydrostatics  Practice  Problem 


A  rectangular  panel ,  3  feet  wide  and  6  feet  long,  is  held  in  a  tank  at  a  depth  of  7  feet,  parallel  to  the  surface.  An  open- 
air  U-shaped  manometer  is  tapped  at  the  bottom  of  the  tank. 

REQUIREMENT 

(d)  What  is  the  atmospheric  pressure  in  terms  of  "head"  of  water?  (Answer  In  feet.) 


Practice  Requirement  d 

m  34 


(IV)  2.40 


Figure  2:  Example  Practice  Problem  Screen 
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The  post-exam  followed  the  same  format  as  the  pre-exam  and  contained  the  same 
problem  structure  as  shown  in  the  demonstration  and  practice  problems.  The  difference  with  the 
post-exam  was  that  only  one  question  was  asked  from  each  different  section  within  the  module; 
however  this  question  was  identical  conceptually  to  the  practice  and  demonstration  problems. 

After  completing  the  online  post-exam,  the  questions  were  corrected  within  seconds  and 
marked  as  correct  (with  a  green  checkmark)  or  incorrect  (with  a  red  x).  The  final  results 
included  the  percent  correct  on  the  post-exam  and  percent  improvement  from  the  pre-exam  to  the 
post-exam.  Correct  answers  to  incorrect  problems  were  not  provided.  An  email  was  sent  to  the 
account  on  file  stating  the  evaluator  had  successfully  completed  the  given  section  with  the 
indicated  percentage  correct. 

The  evaluated  course  was  heavily  problem  based  with  supporting  theory  added  as 
necessary.  The  course  was  designed  for  the  typical  engineer  preparing  for  the  P.E.  and  geared 
towards  individuals  with  a  technical  background,  ideally  with  an  undergraduate  engineering 
degree.  Although  non-technical  individuals  could  successfully  complete  the  course,  it  would 
probably  require  the  use  of  additional  resources  outside  the  individual  refresher  program.  The 
program  course  was  designed  as  a  review  course  and  not  intended  to  provide  initial  instruction 

6.2  Screen  Layout 

Each  module  contained  a  basic  problem  screen  and  was  separated  into  three  sections: 
problem  statement,  visual  aids,  and  control  menu.  Figures  1  &  2  show  an  example  problem 
screen  with  the  problem  statement  in  the  top  window,  the  visual  aids  on  the  lower  right,  and  the 
control  menu  on  the  lower  left. 

6.2.1  Problem  Statement 

The  problem  statement  screen  listed  the  general  situation,  the  solution  requirement,  four 
multiple-choice  answers,  and  four  command  options.  Within  the  problem  requirements  were 


17 


hypertext  links  that  displayed  the  requested  information  in  the  visual  aids  section.  The  command 
options  that  were  provided  allowed  the  user  to  activate  the  instructor  video  recording  (or  audio  in 
the  online  version),  view  additional  visual  aids  that  were  related  to  the  specific  module,  display 
the  Greek  Alphabet,  or  continue  onto  the  next  problem. 

6.2.2  Visual  Aids 

The  purpose  of  this  screen  was  to  display  supporting  information  for  the  problem.  This 
information  included  basic  equations  relating  to  the  subject  topic,  associated  tables  and  graphs, 
and  video  clips  of  tests  that  were  performed  to  determine  certain  parameters  within  the  problem 
(i.e.  how  to  conduct  a  plasticity  test).  Additionally,  when  the  problem  was  first  introduced,  the 
visual  aids  screen  displayed  a  free  body  diagram  or  a  sketch  of  the  problem  situation  as  shown  in 
Figure  1 . 

6.2.3  Control  Menu 

The  control  menu  provided  the  user  with  a  means  to  navigate  through  the  entire  program, 
and  access  additional  information  about  the  resources  used  to  develop  the  specific  topic  area.  A 
list  of  resources  and  comprehensive  glossary  was  accessed  through  the  “tools”  option. 
Additionally,  one  available  tool  was  an  internal  scientific  calculator,  which  was  capable  of 
performing  very  basic  operations.  Another  key  option  was  the  control  of  the  video  instructor. 

The  evaluator  could  start,  stop,  and  replay  the  video  clips;  however,  once  the  video  clip  began  it 
could  not  be  partially  rewound.  This  caused  the  evaluator  to  replay  the  entire  video  clip  even  if 
only  part  of  the  instruction  was  required. 

6.3  Individual  Module  Evaluation  Ratings 

The  comparison  charts  in  Appendix  B-l  and  B-2  summarize  the  ratings  assigned  to  each 
individual  module  and  is  divided  into  delivery  and  content  related  factors.  Table  3  summarizes 
the  combinations  of  all  7  modules  weight  factors  for  a  specific  rating  factor  and  shows  the 
average  value  of  each  factor  for  the  entire  course.  The  entire  course  received  a  rating  of  76.6%. 
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The  factors  associated  with  the  content  of  the  course  module  are  shown  in  italics,  whereas 
delivery  related  factors  are  in  regular  text. 


Table  3:  Civil  Engineering  Review  Course  Evaluation 


Form  /  Delivery  Method  /  Content 


Face-to-Face  contact  between  students  and  teachers 


Instructor  speaks  clearly  and  slowly  _ 


Fundamental  concepts  reviewed  and  explained^ _ 

Effective  use  of  visual  aids  (informative,  legible,  subject  applicability) 


'echnical  terms  clearly  identified  _ 


Weight 

Factor 


Course  Content  is  comprehensive  _ 


Course  covers  al[  major  topic  areas _ 

Course  ties  al[  major  topic  areas  together _ 

Feedback  is  analytical,  suggestive,  and  timely  (not  just  provisions  of 
the  right  answers) 


bility  to  see  and  hear  the  instructor  _ 


fotive  participation  by  the  learner  vice  passively  listening  to  lectures 
Written  text  provided  that  supplements  course  lectures 


Hyperlinks  provided  for  more  topic  information 


Internal  emphasis  focusing  on  how  instructional  material  relates  to 
everyday  reality 


Concepts  are  presented  in  a  variety  of  contexts  and  expressed  in  a 
variety  of  ways 


Provides  students  with  the  ability  to  verbally  express  ideas  and 

discuss  problems  with  peers  and  teachers _ 

Promote  Group  learning  (Interaction  with  other  students) 


Course  provides  challenging,  but  attainable  learning  tasks _ 

Problem  presents  more  information  than  necessary  to  allow  student 
decide  which  information  is  relevant 


Course  material  presented  in  a  manner  that  increases  students  self 
confidence  (start  with  simple  concepts  before  introducing  new  ones) 


Problems  require  students  to  make  logical  assumptions 


Problems  require  students  to  think  critically  and  analyze  information 


Student  provided  with  a  "hands-on"  model 


Student  discussion  of  relevant  P.E.  experiences 


Introduction  poses  questions  that  are  interesting  and  familiar  to  the 
student 


Entire  CE 
Review 
Course 


100% 

100% 

100% 

91.4% 

100% 

95.7% 

100% 

100% 

100% 

65.7% 

100% 


100%  50% 


100%  50% 


100%  100% 


100%  100% 


100%  100% 


100%  50% 
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The  majority  of  the  factors  listed  in  Appendix  B  concentrate  on  the  delivery  method  used 
to  present  the  material.  Several  factors  concentrate  on  the  individual  content  for  each  module. 
However,  for  the  delivery  method  factors  in  plain  text,  the  ratings  are  relatively  consistent  across 
all  seven  modules  since  the  types  of  visuals,  tools,  resources,  and  screen  layout  were  the  same 
for  each  module.  The  only  major  change  in  presentation  style  was  the  individual  instructor’s 
ability  to  accurately  and  effectively  convey  the  topic  to  the  student.  The  individual  content 
factors  will  be  discussed  in  section  6.4.3  through  6.4.9. 

Referring  to  Appendix  B-l  and  B-2,  a  hypothetical  “ideal”  classroom  was  designed  to  be 
used  as  a  baseline  for  comparison  with  the  evaluated  course.  The  “ideal”  factors  were  assigned  a 
value  of  100%.  In  many  instances,  key  factors  for  a  distance-learning,  self-paced  class 
(specifically  this  CD-ROM  training  course)  were  assigned  a  score  of  0%  due  to  the  manner  in 
which  these  factors  were  selected.  For  example,  some  of  the  factors  that  received  an 
effectiveness  rating  of  0%  included  the  ability  to  promote  group  learning,  the  use  of  a  “hands- 
on”  model,  and  the  opportunity  to  share  personal  experiences  with  similar  problems.  Although 
all  the  factors  were  considered  in  rating  the  course,  they  are  more  applicable  to  a  “new 
instruction”  course  and  not  directly  applicable  to  a  distance  learning  “review”  type  course. 

The  ability  to  have  face-to-face  contact  between  students  and  instructors  was  assigned  a 
value  of  50%.  In  an  “ideal”  classroom,  the  student  is  able  to  ask  questions  of  the  instructor  and 
receive  immediate  feedback.  In  this  course,  the  evaluator  was  able  to  “see”  the  instructor  in  the 
video  presentations,  which  made  it  easier  to  follow  the  instruction. 

During  the  course,  the  evaluator  had  the  option  to  be  actively  involved  in  the  instructions 
by  working  practice  problems  along  with  the  instruction  from  the  demonstration  problem. 
However,  the  evaluator  had  the  option  to  passively  listen  to  the  instructor  and  not  actively 
participate.  As  a  result,  the  active  participation  in  class  factor  for  this  course  was  assigned  a 
constant  value  of  50%  for  each  module  since  the  evaluator  was  not  required  to  participate  during 
the  instruction.  There  were  no  outside  homework  assignments  required  to  complete  the  course. 
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Finally,  the  course  is  designed  to  be  self-paced  which  eliminates  the  ability  to  verbally 
express  ideas  and  discuss  problems  with  peers  and  coworkers.  However  the  factor  was  assigned 
a  rating  of  20%  since  the  typical  engineer  studying  for  the  P.E.  has  the  ability  to  discuss  the 
problems  with  other  engineers  in  the  work  place. 

6.4  Course  and  Module  Impressions 

Overall  the  course  appears  to  be  an  excellent  tool  in  preparing  for  the  Civil  Professional 
Engineer  exam.  The  entire  course  was  presented  in  a  very  professional  manner,  with  the  proper 
emphasis  given  to  each  of  the  seven  major  topic  areas.  Each  module  contained  an  animated 
introduction  slide,  which  provided  a  good  initial  impression  of  the  quality  and  time  involved  in 
developing  each  module.  The  cost  for  the  course  was  reasonable  for  the  amount  and  quality  of 
instructional  content  included.  Based  on  the  previously  identified  selection  criteria,  this  course 
met  all  the  selection  criteria,  and  after  evaluation,  met  the  expectations  of  the  evaluator. 

6.4.1  Overall  Advantages 

The  layout  of  the  course  was  very  user  friendly.  The  main  problem  screens  were  easy  to 
navigate  through  and  provided  all  the  necessary  information  to  complete  each  problem  through 
the  use  of  hyperlinks  and  the  available  tools;  therefore  no  additional  reference  material  was 
needed  to  successfully  complete  the  problems.  All  required  tables  and  figures  were  included 
within  the  program  itself. 

The  course  instruction  was  presented  in  a  clear  and  concise  manner,  which  made  it  easy 
to  review  the  topic  even  with  the  absence  of  the  physical  instructor.  Problems  were  presented  in 
a  logical  manner,  and  few  questions  arose  that  were  not  covered  by  another  section  of  the 
module.  In  a  few  instances,  the  evaluator  identified  questions  from  unclear  concepts  during  the 
theory  discussion,  but  no  questions  arose  as  to  the  intent  of  the  review  problems  or  how  to  go 
about  solving  the  problem.  When  a  clarification  was  needed,  answers  were  obtained  from  either 
Keepsmart  Engineering  or  the  instructor  within  48  hours. 
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The  ability  to  print  out  the  solution  to  each  demonstration  problem  was  a  great  reference 
to  be  used  during  the  P.E.  exam.  Additionally,  the  internal,  scientific  calculator  provided  a  good 
resource  and  tool  in  calculating  problems  and  allows  the  program  to  be  a  self-sufficient  training 
course.  The  calculator  contained  specific  help  options  that  identified  and  defined  each  key.  A 
final  benefit  of  this  course  was  the  email  message  that  indicated  the  evaluator’s  completion  of  a 
specific  module.  This  email  provided  existing  registered  Professional  Engineers  with  credit  for 
annual  continuing  education  requirements. 

6.4.2  Overall  Disadvantages 

To  assist  the  IES  in  creating  a  separate,  competitive  program,  several  areas  were 
identified  in  the  evaluated  course  that  would  improve  the  course  and  add  greater  benefit  future 
students.  These  areas  were  identified  based  on  the  ideal  learning  environment  factors  discussed 
in  section  5.3. 

First,  as  identified  in  course  evaluation  factors,  the  “ideal”  learning  environment  would 
include  a  text  associated  with  the  course  material.  A  printable  solutions  option  for  the 
demonstration  problems  was  available  with  this  program  through  the  use  of  Adobe  Acrobat(C). 
The  problem  was  with  printing  a  demonstration  or  practice  problem  to  be  used  as  a  worksheet  for 
completing  the  problem.  In  this  case,  the  page  could  not  be  formatted  prior  to  printing  and  the 
problem  printed  on  a  single  page.  As  a  result,  problems  with  multiple  questions  were  printed  in 
a  very  small  font  size.  This  made  it  difficult  to  read  and  did  not  provide  sufficient  room  to  work 
out  the  individual  questions.  As  an  improvement,  the  problem  should  be  able  to  be  formatted  as 
necessary  to  increase  the  font  size  and  provide  sufficient  room  to  complete  all  question 
calculations. 

One  measure  of  an  ideal  classroom  is  whether  the  concepts  are  presented  in  a  variety  of 
contexts  and  expressed  in  a  variety  of  ways.  The  problems  presented  in  this  course  were 
repetitive.  As  previously  mentioned,  all  four  problems  (pre-exam,  demonstration,  practice,  and 
post-exam)  were  formatted  in  exactly  the  same  manner  with  the  exception  of  the  actual  values 
within  the  problem.  The  course  teaches  how  to  complete  a  specific  type  of  problem  and  did  not 
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expose  the  evaluator  to  other  methods  of  presenting  the  problem.  Additional  practice  problems 
should  be  added  as  a  supplement  to  the  course,  which  covers  the  same  topics  already  presented, 
but  provides  the  evaluator  with  the  ability  to  see  a  problem  presented  in  a  different  manner. 

As  indicated  in  Table  3,  in  order  to  be  effective,  feedback  during  a  course  needs  to  be 
analytical,  suggestive,  and  timely.  Typically,  the  response  to  an  incorrect  answer  was  that  it  was 
wrong  and  did  not  indicate  why  it  was  wrong.  Similarly,  correct  answers  were  not  quantified, 
only  indicated  as  correct.  For  example,  in  an  economics  question  involving  the  comparison  of 
two  or  more  companies,  the  question  asked  which  company  had  the  highest  rate  of  return.  The 
answers  were  company  A  or  B.  When  the  correct  answer  was  selected,  the  instructor  responded 
with  an  audio  clip  like,  “Correct,  Company  A  has  the  higher  rate  of  return.”  The  actual 
calculated  rate  of  return  should  be  provided  to  serve  as  confirmation  that  the  problem  was 
calculated  correctly  and  not  just  arrived  at  by  chance.  The  audio  clip  should  respond  like, 
“Correct,  the  ROI  for  Company  A  is  18%.”  Similarly,  if  the  answer  is  incorrect,  the  audio  clip 
should  explain  why  that  answer  is  not  the  correct  answer. 

In  order  to  understand  the  topics  being  presented,  each  term  needs  to  be  clearly  defined, 
or  the  ability  to  define  the  term  should  be  made  available  to  the  student.  This  course  provides  an 
extensive  technical  library  that  is  accessed  via  hyperlinks  throughout  the  problem  text.  Although 
the  definitions  listed  are  very  helpful,  there  is  no  search  engine  included  that  allows  the  student 
to  look-up  specific  terms.  As  a  convenience  to  the  student,  the  ability  to  enter  a  technical  term 
and  obtain  its  definition  is  not  necessary  but  would  be  an  added  benefit  to  the  program. 

Finally,  once  the  video  instructor  is  activated,  the  evaluator  was  required  to  listen  to  the 
entire  pre-recorded  lecture  segment,  which  could  last  between  10  and  60  seconds.  There  was  not 
an  option  to  partially  rewind  or  fast-forward  the  instruction.  This  proved  to  be  an  inconvenience 
when  only  a  section  of  the  instruction  was  needed  to  clarify  the  problem  for  the  student. 
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6.4.3  Economics 


Although  there  is  not  a  specific  economics  section  on  the  new  breadth  and  depth  Civil 
P.E.  Exam,  the  concepts  are  important  to  understand  and  may  appear  within  one  of  the  five 
major  Civil  Engineering  categories.  The  economics  module  is  divided  into  7  separate  sections 
that  cover  the  topic  areas  shown  in  Table  2.  Fundamental  concept  coverage,  feedback,  how 
material  relates  to  everyday  reality,  and  concepts  being  presented  in  a  variety  of  manners  are  all 
evaluation  factors  that  address  individual  module  content. 

The  feedback  for  economics  received  a  rating  of  50%  since  many  of  the  answers  were  not 
quantified  by  providing  the  student  with  the  correct  number.  Additionally,  the  module  did  not 
explain  why  the  other  answers  were  are  incorrect,  which  makes  it  difficult  to  determine  where 
the  error  in  the  calculations  occurred. 

How  the  economics  problems  related  to  reality  received  a  rating  of  50%  primarily 
because  each  topic  was  presented  as  a  textbook  problem  with  little  reference  to  how  it  applies  to 
a  real-life  situation.  Additionally,  each  topic  was  presented  in  one  manner.  There  was  no  variety 
between  demo  and  practice  problems.  Finally,  each  of  the  topics  in  the  module  presented  only 
the  information  necessary  to  solve  the  problems  which  eliminated  the  need  for  the  student  to 
decipher  the  important  values.  As  a  result,  this  area  was  given  a  value  of  80%. 

6.4.4  Geotechnical 

The  entire  Geotechnical  topic  was  covered  very  extensively  and  thoroughly.  Under  the 
Soil  Classification  section,  there  was  a  detailed  video  clip  of  how  to  perform  a  liquid  and  plastic 
limit  tests.  This  video  was  informative  but  added  little  value  to  understanding  how  to  complete 
the  problems.  Conversely,  the  video  clip  provided  in  the  consolidation  section  was  very 
beneficial  in  understanding  the  consolidation  theory.  It  clearly  defined  the  theory  and 
demonstrated  its  application. 


The  individual  values  assigned  to  the  Geotechnical  evaluation  factors  were  higher  than 
the  economics  section.  Compared  to  available  reference  material,  the  topic  appeared  to 
completely  comprehensive  and  covered  all  major  topic  areas.  Additionally,  the  feedback  on 
correct  and  incorrect  answers  was  more  qualitative  and  provided  the  actual  value  corresponding 
to  the  answer  which  caused  the  feedback  rating  to  be  higher  at  70%.  This  value  was  not  assigned 
a  100%  value  since  100%  represents  real-time  feedback  from  an  instructor  in  a  classroom 
setting.  Finally,  the  problems  contained  information  in  excess  of  that  necessary  to  complete  the 
problem  which  caused  this  factor  to  be  100%. 

6.4.5  Hydrology  and  Hydraulics 

The  module  provided  an  extensive  review  of  the  topic.  The  sections  discussing  closed 
circuit,  friction  loss,  minor  losses,  and  momentum  were  very  well  integrated.  However,  the 
module  needed  to  include  more  discussion  on  pumps  and  turbines,  reservoir  sizing,  and 
Bernoulli’s  equation.  As  a  result,  the  comprehensiveness  of  the  course  only  received  an  80%. 
The  rest  of  the  individual  evaluation  scores  were  similar  to  the  Geotechnical  section  with  the 
exception  of  problem  information.  This  factor  received  a  value  of  80%  because  no  excess 
information  was  provided  in  the  problem. 

6.4.6  Structures 

Structures  is  a  very  broad  topic  and  difficult  to  cover  all  the  specific  aspects  of  structural 
design  in  a  short  refresher  course.  The  module  covered  5  major  topics,  but  should  have  included 
several  additional  topics  that  are  discussed  in  section  7.0.  As  a  result,  the  comprehensiveness  of 
the  module  received  a  rating  of  70%.  The  other  rating  factors  were  identical  to  the  Hydraulics 
and  hydrology  module. 

During  the  instruction  of  this  module,  the  instructor  discussed  the  process  behind  solving 
the  problem  rather  than  immediately  discussing  which  formulas  to  use  for  the  problem  solution. 
This  brief  process  introduction  was  very  beneficial  in  understanding  how  to  approach  each 
problem.  The  drawback  to  this  module  was  the  lack  of  requirements  for  each  problem.  Many  of 
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the  demonstration  problems  only  contained  3  requirements  with  only  1  of  those  requirements 
being  asked  in  the  practice  problem. 

6.4.7  Environmental  and  Sanitation 

This  module  was  very  comprehensive  and  covered  all  the  major  topic  areas  completely 
which  caused  the  comprehensiveness  factor  to  be  assigned  a  value  of  90%.  The  ratings  for  this 
module  were  the  same  as  economics  except  the  comprehensiveness  factor.  The  benefit  of  this 
module  over  the  others  was  that  it  clearly  quantified  correct  and  incorrect  answers  so  the 
evaluator  could  determine  where  the  mistake  was  made  in  the  calculations. 

6.4.8  Transportation 

Thorough  coverage  of  all  topic  areas  with  the  exception  of  specific  roadway  design 
which  resulted  in  a  rating  of  90%  for  the  module  comprehensiveness.  Additionally,  the  feedback 
given  for  each  problem,  as  well  as  the  remainder  of  the  evaluation  factor  ratings,  mirrored  the 
Economics  section  in  that  it  did  not  quantify  “yes”,  “no”,  “true”,  and  “false”  answers.  If 
answered  incorrectly,  there  was  no  explanation  at  why  that  was  the  incorrect  answer.  This 
module  also  contained  several  video  clips  that  showed  how  traffic  lights  were  timed,  how  lane 
sizing  was  accomplished,  and  how  an  Automatic  Traffic  Recorder  functioned.  Although 
informative,  these  video  clips  were  not  necessary  to  fully  understand  these  concepts. 

6.4.9  Surveying 

Although  there  is  not  an  afternoon  section  on  the  P.E.  exam  dedicated  to  surveying,  the 
concepts  in  this  module  are  incorporated  into  the  5  main  topic  areas.  This  entire  module  was 
extremely  comprehensive  and  provided  the  best  feedback  of  any  of  the  other  6  modules.  When 
an  answer  was  incorrect,  the  instructor  indicated  where  in  the  calculation  the  error  had  most 
likely  occurred.  Additionally,  each  correct  and  incorrect  answer  was  quantified  which  resulted 
in  the  feedback  factor  being  assigned  a  rating  of  80%.  The  remaining  evaluation  factors  received 
ratings  identical  to  the  Geotechnical  module. 
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7.  COST  ANALYSIS 


7.1  General 


A  break  even  analysis  was  completed  to  determine  whether  the  creation  of  a  competitive 
product  will  be  cost  beneficial  to  the  IES.  According  to  the  NCEES,  an  estimated  12,000 
engineers  take  the  Civil  P.E.  Exam  annually.  This  break  even  analysis  assumes  a  linear  sales 
growth  to  achieve  10%  and  20%  market  shares  in  the  U.S.  market  for  distance  learning  products 
of  this  type. 

7.2  Product  Costs 

The  cost  to  develop  a  new  course  includes  both  fixed  and  variable  costs.  Fixed  costs 
include  all  costs  that  must  be  paid  regardless  of  the  amount  of  product  sold.  In  this  analysis,  all 
the  production  and  marketing  costs  were  considered  fixed  costs.  Variable  costs  are  dependent  on 
the  number  of  units  sold  and  are  typically  linear  in  nature.  The  combination  of  fixed  and 
variable  costs  are  added  to  obtain  the  total  cost  of  the  product.  Table  4  summarizes  the  fixed  and 
variable  costs  for  this  study.  Most  of  the  costs  are  self  explanatory. 

A  previous  analysis  suggests  that  the  addition  of  an  “on-demand”  instructor  would  be  an 
attractive  service  to  students  purchasing  the  IES  product.  For  this,  the  cost  of  a  dedicated  help 
line  was  included  at  an  assumed  rate  of  $  150/hour.  Additionally,  only  30  minutes  of  questions 
was  estimated  for  every  10  units  sold.  This  equated  to  $7.50  for  product  support  per  unit  sold. 
The  waste  factor  takes  into  account  damage  CDs  and  errors  in  reproducing  both  the  text  and  CD 
information. 
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Table  4:  Total  IES  Product  Costs 


Unit  Price 

Total  Price 

Fixed  Costs: 

Development 

IES  Media  Designers 

5000 

hrs 

$23 

$115,000 

IES  Instructor  Development 

$40 

$200,000 

IES  Project  Management 

80 

$40 

$3,200 

Web  Page  Development 

40 

$25 

$1,000 

Marketing 

Develop  Printed  materials 

80 

hrs 

$25 

$2,000 

Print  and  mail  Postcards 

24,000 

Ea 

$0.50 

$12,000 

Total  Fixed  Costs: 

$333,200 

Variable  Costs: 

Per  Unit  Costs: 

Compact  Discs 

2 

Ea 

$2.00 

$4 

Computer  Copying  Time 

0.5 

hours 

$25 

$13 

Shipping 

1 

LS 

$10 

$10 

Admin  Support 

0.5 

hrs 

$14 

$7 

Dedicated  Help  Line 

0.5 

hrs 

$15 

$8 

1  (10  units  =  0.5  hours  help) 

Printed  text  Reproduction 

1 

LS 

$10 

$10 

Waste  (10%) 

1 

LS 

$1 

$1 

Instructor  Royalties  (1%) 

1 

LS 

$1 

$1 

Total  Variable  Costs  /  Unit 

$53 

7.3  Break  Even  Analysis 

There  is  evidence  that  engineers  preparing  to  take  the  P.E.  examination  are  moderately 
sensitive  to  course  prices.  Based  on  a  survey  conducted  by  IES,  20  out  of  27  engineers  would 
pay  up  to  $500  for  a  refresher  course.  This  analysis  considered  selling  prices  between  $200  and 
$500.  As  shown  in  Figure  3,  if  the  selling  price  was  $300,  the  IES  would  need  to  sell 
approximately  1350  units  before  making  a  profit.  Similarly,  at  $400,  the  IES  would  only  need  to 
sell  950  units  break  even. 
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Figure  3:  EES  Break  Even  Analysis 

The  target  market  for  the  product  is  nationwide  with  the  potential  to  reach  internationally. 
Data  from  NCEES  indicates  that  in  2001,  the  number  of  North  American  engineers  applying  for 
the  exam  was  approximately  12,000.  This  represented  an  increase  of  almost  1,000  applicants 
throughout  the  country  from  last  year.  This  growth  is  expected  to  continue  in  the  future. 

The  IES  product  is  likely  to  obtain  a  small  share  of  the  market  shortly  after  launch.  This 
share  is  likely  to  increase  with  time.  Two  assumptions  are  made  in  the  market  share  growth. 
First,  a  pessimistic  growth  rate  of  2%  a  year  for  5  years  will  yield  a  total  market  share  of  10% 
and  would  require  2.0  years  before  IES  would  make  a  profit  as  shown  in  Figure  4.  The  second 
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assumption  is  an  optimistic  growth  rate  of  4%  a  year  for  5  years  to  obtain  a  total  market  share  of 
20%.  In  this  case,  the  break  even  point  would  occur  at  2.8  years  as  shown  in  Figure  5.  In  both 
instances,  a  constant  market  of  12,000  engineers  nationwide  and  a  selling  price  of  $3 00/unit  was 
considered. 


Figure  4:  Break  Even  Time  based  on  2%  Annual  Growth 


♦  $300  Selling  prce 

- Fixed  Cost 

A  Total  Costs 


Figure  5:  Break  Even  Time  based  on  4%  Annual  Growth 


7.4  Results 


Engineers  preparing  for  the  P.E.  exam  are  only  moderately  price  sensitive.  Addtionally, 
if  available  P.E.  distance  learning  courses  are  priced  relatively  high,  the  costs  may  be  beyond  the 
limit  acceptable  by  engineers.  Therefore,  by  maintaining  the  selling  price  of  the  product  below 
the  competitors,  this  new  product  will  be  more  marketable.  The  current  market  price  for  a 
typical  CD-ROM  refresher  course  is  approximately  $500.  By  selling  the  product  at  $300  per 
unit,  the  expected  sales  growth  will  be  greater  than  selling  for  a  higher  price.  Additionally,  a 
profit  will  be  made  from  the  product  around  4  years  after  initial  introduction. 
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8.  CONCLUSIONS 


Professional  Engineer  review  courses  can  be  effectively  developed  and  delivered  through 
means  of  a  distance  learning  format.  There  is  a  market  for  a  distance  learning  P.E.  review 
course.  Engineers  preparing  for  the  exam  will  consider  purchasing  a  distance  learning  review 
class  course.  A  survey  conducted  by  the  IES  indicated  that  88%  of  engineers  in  the  State  of 
North  Carolina  would  purchase  a  distance  learning  refresher  course.  Of  these  engineers,  82% 
would  purchase  a  combined  CD-ROM  and  textbook  refresher  course. 

Several  distance  learning  options  are  currently  available.  The  on-line/offline  combined 
CD-ROM  course  developed  by  ASCE  and  marketed  by  Keepsmart  Engineering  was  an  excellent 
program  and  proved  to  be  an  adequate  benchmark  for  IES  for  future  course  development. 

Profession  Engineer  examinees  are  price  sensitive  and  cost  is  an  issue  to  be  considered  in 
course  development.  Most  engineers  would  only  pay  up  to  $500  for  a  quality,  comprehensive 
refresher  course. 

The  overall  evaluated  course  provided  a  very  effective  tool  in  reviewing  for  the  P.E. 
exam  and  is  recommended  for  any  Civil  Engineer  preparing  for  the  licensure  exam.  The  course 
was  very  “user-friendly”  and  easily  navigable.  The  screen  layout  and  the  ability  to  have  the 
instructor  explain  how  to  complete  the  individual  problem  requirements  were  two  benefits  of  this 
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9.  RECOMMENDATIONS 


A  review  of  the  ASCE  refresher  course  content  revealed  several  concepts  and  items  that 
should  be  included  and  considered  in  developing  a  new  course,  such  as: 

•  Ease  of  navigating  through  the  course 

•  Comprehensive  topical  coverage 

•  Video  and  audio  clips  of  instructors  explaining  the  topic 

•  Explanation  of  the  process  involved  in  solving  a  problem 

•  Effective  use  of  visual  aides  and  graphics  to  fully  explain  the  problem 
(i.e.  Freebody  diagrams) 

•  A  diverse  variety  of  problems 

•  Strong  imbedded  scientific  calculator 

•  Quantitative  feedback  on  problems 

•  Dedicated  help  line  to  answer  basic  questions 

The  cost  analysis  in  section  7  included  variable  costs  for  required  student  help.  To 
minimize  this  cost,  it  is  recommended  that  an  online  bulletin  board  be  established  as  the  primary 
means  of  answering  and  asking  questions.  This  would  allow  students  to  share  solutions  to 
practice  problems  and  greatly  reduce  the  direct  interaction  time  of  the  instructor  or  IES 
personnel. 

The  designed  length  of  the  evaluated  course  was  sufficient  for  the  information  given. 
However,  the  course  could  be  lengthened  without  reducing  the  desirability  of  the  course,  by 
adding  additional  practice  problems.  Each  individual  module  took  between  4  and  10  hours  to 
complete.  This  time  was  also  determined  to  be  adequate  for  reviewing  a  given  section.  Overall 
a  new  course  should  be  designed  to  be  completed  in  less  than  1 00  hours,  provided  additional 
practice  problems  and  the  subject  areas  discussed  below  are  included. 
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The  evaluated  course  contained  several  video  clips  that  displayed  tests  or  demonstrated 
the  theory  behind  the  problem.  Although  graphics  and  diagrams  are  important  in  an  Engineering 
class,  these  items  can  be  removed  from  the  review  course.  For  example,  video  of  how  water 
flows  through  a  pipe  requires  time  to  develop  the  graphic,  but  adds  little  to  understanding  the 
concept.  Free-body  diagrams  that  are  not  animated  should  still  be  used  to  explain  the  problem. 

When  explaining  how  to  complete  a  problem,  the  process  should  be  discussed  prior  to 
beginning  the  solution,  similar  to  the  structures  module.  This  will  help  the  student  understand 
how  to  go  about  solving  the  problem  and  give  the  student  an  idea  of  how  much  work  is  involved 
to  obtain  the  correct  solution. 

A  new  course  should  consist  of  the  same  7  major  topic  areas  that  were  covered  in  the 
evaluated  course.  In  addition  to  the  subject  already  covered  under  each  module  and  as  listed  in 
Table  1,  the  subject  matters  listed  in  Table  5  should  be  added  or  discussed  more  in-depth.  These 
items  were  identified  in  Reference  8,  but  were  not  directly  covered  in  the  refresher  course. 

The  format  used  in  presenting  the  material  needs  to  be  user  friendly  and  easy  to  navigate. 
The  example  screens  shown  in  Figures  1  and  2  are  a  good  baseline  for  design.  At  any  given  time 
through  the  problem  discussion,  the  student  should  be  able  to  view  the  free-body  diagram  and  the 
problem  statement. 

The  tools  section  of  the  evaluated  program  was  very  useful.  Specifically,  the  calculator 
allowed  the  program  to  stand-alone.  However  the  imbedded  calculator  should  be  redesigned  to 
handle  multiple  calculations  at  one  time.  As  a  recommendation,  the  calculator  should  be 
modified  to  provide  a  “( )”  or  similar  function  to  allow  for  long  strings  of  calculations  to  be 
entered.  This  recommendation  is  based  on  personal  preference  with  using  calculators  and  is 
partially  substituted  by  the  inclusion  of  a  memory  addition,  subtraction,  and  storing  capability. 

As  a  follow-on  to  this  project,  and  prior  to  designing  a  competitive  product,  a  complete 
market  analysis  needs  to  be  completed  to  identify  the  number  of  engineers  throughout  the 
country  that  have  used  refresher  courses  in  the  past  and  estimate  the  expected  usage  in  the  future. 
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Additionally,  the  expected  selling  price,  revenue,  and  payback  period  should  be  identified  to 
determine  how  effective  a  competitive  course  would  be  in  the  marketplace  prior  to  beginning  the 
development  of  a  new  course. 


Table  5:  Additional  Subjects  to  be  Discussed 


Add 

Economics 

Break  Even  Analysis 

Geotechnical 

More  Discussion  on  Phase  Diagrams 

Mohr’s  Circle  review 

Surcharge  Loading 

Hydraulics  and 
Hydrology 

Review  of  Bemoulii’s  and  Energy 
Equations 

Pumps  and  Turbine  Efficiencies  and 
Horsepower 

Reservoir  Yield  and  Sizing 

Structures 

Stress  /  Strain  Relationship  Review 

Non-Cantilever  Beams 

Indeterminate  Structures 

Anchoring 

T-Beams 

Pre-Stressed  /  Post  Tensioned 

Footings 

Different  Connections  (Welded,  Bolted) 

Sanitary  Sewer  and 
Environmental 

More  Discussion  on  Water  Chemistry 
Calculations 

Transportation 

Roadway  Banking  Design 

Pavement  Design 

Parking  Design 

Surveying 

No  Additions  Recommended 

An  additional  recommendation  to  justify  the  ratings  assigned  to  each  “learning” 
environment  factor  in  the  evaluated  course,  a  separate  survey  should  be  performed  on  individuals 
that  have  participated  in  distance  learning  classes  to  determine  the  effectiveness  of  each  factor. 
This  survey  should  be  completed  prior  to  developing  a  new  course  to  determine  the  course’s 
marketability. 
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APPENDIX  A:  COURSE  WEBSITE  REFERENCES 


Video  Tape  claudia  baker@ncsu.edu  www.ies.ncsu.edu 


APPENDIX  B-l:  PRIMARY  DISTANCE  LEARNING  COURSE  EVALUATION  FACTORS  AND  RANKINGS 


I 


Note:  Rows  in  Italics  represent  “content”  factors.  All  other  rows  represent  “delivery”  factors. 


APPENDIX  C:  COMPARISON  OF  PRODUCT  FEATURES 


Number  across  the  top  of  the  row  shows  maximum  amount  of  points  allowed  for  each  major  category. 
Numbers  in  the  individual  boxes  represents  the  amount  of  point  given  for  that  specific  area. 

Courses  Sponsored  by  ASCE,  NCEES,  or  NSPE  received  higher  marks. 


